Re-work this paragraph. Motivations should start by stating what the problem is you're trying to solve (sending false positive notifications to authors about builds/tests already failing at head). Once the problem is explained (including WHY it's important/impactful to solve this problem), then you explain how you plan to solve it.

Also, the phrase "for certain types of automation" is so vague as to be virtually useless. I would either add more details or omit it.

Reworked the paragraph to talk about what we want to do in the end at the top and then go into the how.

Removed the vague wording.

You should also mention that the current postcommit buildbots don't run separate instances for each commit -- they bundle multiple commits together.

This depends upon the configuration and bundling commits is explicitly discouraged in https://llvm.org/docs/HowToAddABuilder.html. I've added some text to describe this.

"a similar configuration to the one run premerge" => "the same configuration used for premerge testing (for PR commits); for direct git commit commits, it will use that same configuration that would have been used with PR testing, if the commit had gone through a PR."

Replace the sentence "The test configuration is intended to be close to the premerge configuration but will be different in some key ways." with:
"The post submit testing is intended to be very close to the premerge testing, but will be different in some key ways."

I don't think the distinction between directly pushed commits/commits from PRs is useful here. I've changed up the wording to say the commit will be tested as if it is going through the premerge pipeline, with some minor differences.

Ack on the second point, changed up the wording.

"Github autoscales" => "Github uses to autoscale".

Nit: "This allows...also allows...This allows..." Might read better if you didn't use 'this allows" so frequently in such a short space. (Some alternatives: "enables", "lets us")

Also I think you mean the same mechanism that LLVM testing on GitHub uses, not GitHub the company specifically uses it.

Clarified to Github ARC so that it's clear it's the software running on our cluster.

First sentence. "The actual testing configuration will be the same as that used by the premerge testing."

I'm using testing configuration to include the set of projects that we're testing (which will be different) and the environment.

The environment should be as close as possible, but I don't think there's any easy way to guarantee that it's the same.

line 107 should become:

"testing. In the postcommit testing, we propose testing all projects"

I've updated the text slightly to make it a bit more clear.

I believe you were trying to get at replacing configuration with testing? I think configuration is more accurate here and more consistent with the rest of the paragraph (especially after adding the definition at the beginning).

I can change it up if it is inhibiting clarity though.

THe MLIR example needs clarification: 1). You need to make it clear that you're talking about postsubmit testing only. 2). You need to explain more about turning red/green -- that reach postsubmit run will mark all tests/builds as either red or green, and THAT is why you need to run all the tests every time: to avoid passing in one part from incorrectly marking failures in a different part as passing when they're not.

At least that's what I think your example is trying to explain, but it's too terse to make that clear to someone who's not already familiar with this.

The sentence "We also explicitly do not test certain projects..." I think that sentence is referring to premerge testing rather than postsubmit? But you need` to make that clear. Also this becomes as run-on sentence. I'd take the last phrase and make it a separate sentence.

Rewrote the example to better explain the exact sequence of events and the logic controlling them. I can add some background on buildbot notifications in the beginning if that's helpful, but I hope the new example should make things clear enough.

The default assumption for this document is going to be that "LLVM" refers to upstream llvm, so you can just say "testing" I think.

It's a bit redundant, but I'd prefer to keep it in there. There will probably be internal (to Google) audiences reading this who might not be as familiar with everything upstream and we have a lot of downstream CI, so keeping it explicit might make it more clear for someone and only be redundant for someone else.

These are some very large paragraphs. The content is fine but a line break between each "point" made would make it easier to read through.

Also I think you mean the same mechanism that LLVM testing on GitHub uses, not GitHub the company specifically uses it.

Recommendation: Add a sentence at the end of this paragraph about the premerge advisor (which you reference in the Data Storage section as if it had been mentioned previously). Something like "We can then use this data to implement a premerge advisor, to help prevent sending false positive test failure notifications to users."